1. Field of the Invention
The present invention relates to computer hardware and software systems and, more particularly, to recovery or restoration of data for such a system in the event of a crash of the system or a disaster which causes the system to become inoperative for a period of time. When such a system crashes or becomes inoperative, data may be lost unless measures have been provided to recover or restore data. Specifically, the present invention provides methods and apparatus which implement substantially real-time networked disk, or data, mirroring over local area networks (LANs) and wide area networks (WANs) in a computer system, such as a SPARC Solaris 2.X environment, for disaster recovery and other applications.
2. Related Art
Various techniques are known for recovery or restoration of data in the event of a crash of a computer system or a disaster which causes the computer system to become inoperative for an indefinite period of time or even permanently. One technique that is known is to replicate data as the data is generated by an application program being executed by the computer system. This technique is typically referred to as disk, or data, mirroring.
Heretofore, data mirroring has been achieved by one of several approaches. One approach is to provide local data mirroring utilizing redundant arrays of independent disks (RAID). Using the RAID approach, data generated by execution of an application program is written to multiple storage devices, such as conventional disk drive devices, contemporaneously with storage of the data on a local input/output (I/O) data storage device. Another approach is to provide volume management software and a redundant storage device on which data is replicated. The volume management software replicates data on the redundant storage device contemporaneously with storage of the data on the local I/O data storage device. Both of these approaches typically provide synchronous data mirroring and are characterized by miniscule delay in the replication of data for system recovery.
Considered in more detail, both RAID and volume management approaches typically provide synchronous versus asynchronous disk mirroring. In a synchronous disk mirroring architecture, such as provided by a RAID or volume management approach, disk updates are committed to each of the disk devices in the mirror before control is returned to the application program. In the event that one of the disks goes out of service, the data is still available on one of the other disk devices in the mirror.
The RAID and volume management approaches can be implemented to protect data locally. While these approaches are satisfactory for local disk mirroring for data recovery in the event of a local I/O disk failure or temporary system crash, they do not address the problem of catastrophic system failure or disaster which renders the computer system inoperative for an extended period of time or even permanently.
Another approach is to provide remote data mirroring in addition to local data mirroring. Using this approach, a remote data mirroring system is implemented both locally and remotely so that data generated locally by execution of an application program is additionally communicated over a network to a remote location for replication. Typically, remote data mirroring enables recovery of the local computer system in the event of a temporary outage or, alternatively, transfer of data processing operations to a remote computer system if the local computer system is not able to recover, the outage is for a prolonged period of time, or a disaster permanently disables the local computer system. Remote data mirroring systems have been commercialized by companies such as international Business Machines, Digital Equipment Corporation, and Data General Corporation in the past. Such remote data mirroring systems are operable in one of several modes, including a synchronous mode, asynchronous mode, and near synchronous mode.
Unfortunately, implementing synchronous data mirroring over a network raises serious performance problems. Rather than working with local data channels that can accept data at 5, 20, or 40 megabytes (MB) per second or higher, the data must travel over a much lower bandwidth channel, stretching out data transfer times. Network latencies pile up on top of the much lower bandwidth, further slowing I/O turnaround times. Any practical experience with an I/O rich application program that has compared network file system (NFS) update performance over local disk performance readily illustrates this point. If networked disk mirroring is implemented using synchronous I/O techniques, application performance is tremendously degraded.
On the other hand, implementing asynchronous disk mirroring over a network raises data integrity problems. In the event of a disaster, the data on the remote, or secondary, computer system may be up to several seconds older than what would be found on the local, or primary, computer system.
The near synchronous mode is a forced compromise between the synchronous and asynchronous modes. Near synchronous data mirroring provides asynchronous remote data mirroring at a preset interval, but requires the local computer system to periodically halt execution of the application program at the preset interval until data replication by the remote computer system is acknowledged.
Therefore, a remote data mirroring system which comprises an architecture configured for optimal data mirroring is needed. Furthermore, such a system is needed which addresses the problem of the limited bandwidth of a network for communication for data over the network.
The present invention provides methods and apparatus for a novel synchronous, asynchronous, or near synchronous computer system remote disk, or data, mirroring system over a network. Fundamentally, the computer network remote data mirroring system in accordance with the present invention comprises an architecture to perform a data update both to a local data device and to a local, chronologically sequenced journal storage area, or writelog device. In one embodiment, the writelog device comprises a redundant data storage device, such as a disk drive device. In another embodiment, the writelog device comprises cache memory and a dirty bit map disk drive device to which data can be written from the cache memory to avoid a memory overflow condition. Advantageously, the device driver for the local data device and the writelog device is layered on top of the operating system environment, so that the computer network remote data mirroring system of the present invention ports to many commercially available computer systems. Once written to the local data device and the writelog device, I/O operation returns control to the application. This delivers to the application I/O performance comparable to simple local disk mirroring data mirroring system of the present invention ports to many commercially available computer systems.
Once written to the local data device and the writelog device, I/O operation returns control to the application. This delivers to the application I/O performance comparable to simple local disk mirroring.
A primary mirror daemon on the local, or primary, computer system monitors the writelog device for data updates and feeds the data over a network in the same order in which it is stored to a receiving remote mirror daemon on a remote, or secondary, computer system, which in turn commits the data updates to a mirror device. Advantageously, the computer network remote data mirroring system in accordance with the present invention operates over different network configurations and is compatible with many different local and remote disk storage devices.
In accordance with the present invention, the writelog device is configured so that more memory space is dynamically assigned to the writing device to prevent a memory overflow condition which might otherwise corrupt stored data. In the embodiment in which the writelog device comprises a disk drive device, additional disk storage is dynamically assigned or another disk storage device is chained into the local, or primary, computer system. In the embodiment in which the writelog device comprises cache memory and a dirty bit map disk drive device, additional disk storage is dynamically assigned or another disk drive is chained into the local, or primary, computer system to prevent memory overflow.
Also in accordance with the present invention, the computer network remote data mirroring system can be structured to provide volume grouping, or logical groups. Consequently, data at the local, or primary, site can be replicated at a plurality of remote sites, as compared to known architectures which provide point-to-point (local to a single remote site) data mirroring. Accordingly, the computer network remote data mirroring system of the present invention provides a master primary mirror daemon and associated child primary mirror daemons, as well as a master remote mirror daemon and associated remote mirror daemons, to process data for replication. A graphical user interface is preferably provided by the computer network remote data mirroring system in accordance with the present invention for confirming the logical groups, as well as for monitoring performance of the remote data mirroring system.
The computer network remote data mirroring system of the present invention additionally provides network bandwidth throttling. Bandwidth throttling enables a predetermined portion of the network bandwidth to be assigned to remote data mirroring depending on the time of day or other criteria.
In accordance with the present invention, a method is provided for ensuring data integrity through a systems failure while updates are occurring in parallel to two data storage devices, such as disk drive devices, simultaneously. The disks are simultaneously updated through a device driver. Preferably, each disk is provided with a disk interface, such as a SCSI interface, to enhance reliability and speed of data updates. Consequently, data is written and thus stored substantially simultaneously on both the local data device and in the writelog device. The method in accordance with the present invention accommodates any of three conditions that may arise in the event of a system crash. The first condition is that the same update data has been stored on both the local data device and in the writelog device; the second condition is that the update data was stored on the local data device, but failed to be stored in the writelog device; and the third condition is that the update data was written to the writelog device, but failed to be stored on the local data device. In accordance with the method of the present invention, the current update data is written to the writelog device, while the immediately preceding update is written to the local data device. If the local computer system crashes, upon recovery or re-boot of the computer system, the two most current updates in the writelog device are written to the local data device to assure that the data stored on the local data device is current.
Additionally, in accordance with the present invention, failure recovery with the primary and remote mirror daemons is initiated automatically for certain failures which do not affect the basic operability of the overall computer system. For example, the computer network remote data mirroring system in accordance with the present invention is automatically recovered upon power shutoff of one of the local, or primary, and remote, or secondary, computer systems or temporary loss of the network link.
Preferably, the computer network remote data mirroring system of the present invention operates in an asynchronous mode. Consequently, the primary and remote mirror daemons are able to take advantage of the entire network bandwidth during data transfers, as one would find in a synchronous mode, yet this is performed independently of the application. One drawback is that in the event of a disaster, the data on the secondary computer system may be up to several seconds older than what would be found on the primary computer system. However, this trade-off between application performance and data synchronicity presents the optimal compromise available between the two requirements. Alternatively, the computer network remote data mirroring system in accordance with the present invention can be operated in a synchronous mode to better assure data synchronicity at the expense of application performance. The computer network remote data mirroring system of the present invention also can alternatively be operated in a near synchronous mode to enable adjustment of the trade-off between application performance and data synchronicity.
The computer network remote data mirroring system in accordance with the present invention provides insurance for mission critical data. The computer network remote data mirroring system of the present invention achieves high application performance by implementing asynchronous, synchronous, or near synchronous data mirroring using network bandwidth throttling. It provides substantially real-time data mirroring over LANs and WANs to quickly move data offsite, yet does not impact application performance significantly. In the event of a disaster taking the primary data center out of service for hours or days, operations can be transferred to a secondary site within minutes, operating on an up-to-the-minute copy of the original data set.